Advanced Semantic Segmentation Architectures - Part 2¶
DeepLabV3+¶
DeepLabv3+ extends DeepLabv3 by adding an encoder-decoder framework. The encoder module processes multi-scale contextual information by applying multi-scale dilated convolution, while the decoder module refines the segmentation results along object boundaries.
Dilated Convolution: With dilated convolution, as we go deeper into the network, we can keep the stride constant but have a larger field of view without increasing the number of parameters or the amount of computation. It also allows for larger output feature maps, which is useful for semantic segmentation.
The reason for using Dilated Spatial Pyramid Pooling is that it has been shown that as the sampling rate increases, the number of valid filter weights (i.e., weights applied to the valid feature region, rather than padded zeros) becomes smaller.
Multiple downsampling of a CNN will lead to the resolution of the feature map becoming smaller, resulting in lower prediction accuracy and loss of boundary information in semantic segmentation. Similarly, adding context around a feature helps in segmenting it better, which is done with atrophic convolutions. DeepLabv3+ helps solve these problems.
Atrous rate¶
Atrous Convolution/Dilated Convolution is a tool to refine the effective field of view of the convolution. It modifies the field of view using a parameter called atrous rate. It is a simple yet powerful approach to enlarge the field of view of filters without affecting the computation or the number of parameters.
DeepLabV3+ adds an encoder based on DeepLabV3 to address the previously observed issue of DeepLabV3 taking too long to process high-resolution images. Applying depthwise separable convolution to spatial pyramid pooling and decoder modules results in a faster and stronger encoder-decoder network for semantic segmentation.
Atrous Separable Convolution¶
(a) and (b), Depthwise Separable Convolution: Factorizes a standard convolution into a depthwise convolution followed by a pointwise convolution (i.e., 1×1 convolution), drastically reduces the computation complexity.
(c) Depthwise Atrous Convolution: Atrous convolution is supported on depthwise convolution. And it is found to significantly reduce the computation complexity of the proposed model while maintaining similar (or better) performance. Combining with pointwise convolution, it is the separable atrous convolution.
So, let's implement DeepLabV3+:
In [ ]:
from google.colab import drive
drive.mount('/content/drive')
Mounted at /content/drive
In [ ]:
import glob
import os
import cv2
import numpy as np
import matplotlib.pyplot as plt
from keras.utils import to_categorical
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
from skimage import io
import os, csv
We configure the paths of the RGB image, the mask and the color table of the classes.
In [ ]:
image_path = '/content/drive/MyDrive/Datasets/Cana weeds/sugarcane2.png'
mask_path = '/content/drive/MyDrive/Datasets/Cana weeds/crop6GT.png'
class_path = '/content/drive/MyDrive/Datasets/Cana weeds/class_dict.csv'
Let's open and plot the image and mask:
In [ ]:
rgb_img = io.imread(image_path)
msk_img = io.imread(mask_path)
In [ ]:
plt.figure(figsize=[16,16])
plt.imshow(rgb_img)
plt.axis('off')
Out[ ]:
(-0.5, 5363.5, 6814.5, -0.5)
In [ ]:
plt.figure(figsize=[16,16])
plt.imshow(msk_img)
plt.axis('off')
Out[ ]:
(-0.5, 5363.5, 6814.5, -0.5)
The mask is composed of an RGB image where each color represents a class. We need to categorize and convert the values of these colors to an image of a band with integer values for each class. To do this, we will create some functions to make this easier:
In [ ]:
def get_label_info(csv_path):
"""
Retrieve the class names and label values for the selected dataset.
Must be in CSV format!
# Arguments
csv_path: The file path of the class dictionairy
# Returns
Two lists: one for the class names and the other for the label values
"""
filename, file_extension = os.path.splitext(csv_path)
if not file_extension == ".csv":
return ValueError("File is not a CSV!")
class_names = []
label_values = []
with open(csv_path, 'r') as csvfile:
file_reader = csv.reader(csvfile, delimiter=',')
header = next(file_reader)
for row in file_reader:
class_names.append(row[0])
label_values.append([int(row[1]), int(row[2]), int(row[3])])
# print(class_dict)
return class_names, label_values
In [ ]:
def one_hot_it(label, label_values):
"""
Convert a segmentation image label array to one-hot format
by replacing each pixel value with a vector of length num_classes
# Arguments
label: The 2D array segmentation image label
label_values
# Returns
A 2D array with the same width and hieght as the input, but
with a depth size of num_classes
"""
# st = time.time()
# w = label.shape[0]
# h = label.shape[1]
# num_classes = len(class_dict)
# x = np.zeros([w,h,num_classes])
# unique_labels = sortedlist((class_dict.values()))
# for i in range(0, w):
# for j in range(0, h):
# index = unique_labels.index(list(label[i][j][:]))
# x[i,j,index]=1
# print("Time 1 = ", time.time() - st)
# st = time.time()
# https://stackoverflow.com/questions/46903885/map-rgb-semantic-maps-to-one-hot-encodings-and-vice-versa-in-tensorflow
# https://stackoverflow.com/questions/14859458/how-to-check-if-all-values-in-the-columns-of-a-numpy-matrix-are-the-same
semantic_map = []
for colour in label_values:
# colour_map = np.full((label.shape[0], label.shape[1], label.shape[2]), colour, dtype=int)
equality = np.equal(label, colour)
class_map = np.all(equality, axis = -1)
semantic_map.append(class_map)
semantic_map = np.stack(semantic_map, axis=-1)
# print("Time 2 = ", time.time() - st)
return semantic_map
def reverse_one_hot(image):
"""
Transform a 2D array in one-hot format (depth is num_classes),
to a 2D array with only 1 channel, where each pixel value is
the classified class key.
# Arguments
image: The one-hot format image
# Returns
A 2D array with the same width and hieght as the input, but
with a depth size of 1, where each pixel value is the classified
class key.
"""
# w = image.shape[0]
# h = image.shape[1]
# x = np.zeros([w,h,1])
# for i in range(0, w):
# for j in range(0, h):
# index, value = max(enumerate(image[i, j, :]), key=operator.itemgetter(1))
# x[i, j] = index
x = np.argmax(image, axis = -1)
return x
def colour_code_segmentation(image, label_values):
"""
Given a 1-channel array of class keys, colour code the segmentation results.
# Arguments
image: single channel array where each value represents the class key.
label_values
# Returns
Colour coded image for segmentation visualization
"""
# w = image.shape[0]
# h = image.shape[1]
# x = np.zeros([w,h,3])
# colour_codes = label_values
# for i in range(0, w):
# for j in range(0, h):
# x[i, j, :] = colour_codes[int(image[i, j])]
colour_codes = np.array(label_values)
x = colour_codes[image.astype(int)]
return x
Let's get the classes we have and the value mapping of the mask classes:
In [ ]:
class_names_list, label_values = get_label_info(class_path)
class_names_string = ""
for class_name in class_names_list:
if not class_name == class_names_list[-1]:
class_names_string = class_names_string + class_name + ", "
else:
class_names_string = class_names_string + class_name
In [ ]:
class_names_list
Out[ ]:
['Invalid_area', 'SugarCane', 'Soil', 'Invasive']
In [ ]:
label_values
Out[ ]:
[[0, 0, 0], [0, 255, 0], [255, 0, 0], [255, 255, 0]]
So we convert the mask and 3 channels to 1:
In [ ]:
gt = reverse_one_hot(one_hot_it(msk_img, label_values))
In [ ]:
np.unique(gt)
Out[ ]:
array([0, 1, 2, 3])
In [ ]:
gt.shape
Out[ ]:
(6815, 5364)
In [ ]:
plt.figure(figsize=[16,16])
plt.imshow(gt, cmap='RdYlGn')
plt.axis('off')
Out[ ]:
(-0.5, 5363.5, 6814.5, -0.5)
In [ ]:
gt
Out[ ]:
array([[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]])
Now let's divide the RGB image and the mask into 256x256 patches:
In [ ]:
from os import mkdir
In [ ]:
mkdir('output')
In [ ]:
OUTPUT_DIR = "/content/output"
TARGET_SIZE = 256
In [ ]:
k = 0
for y in range(0, rgb_img.shape[0], TARGET_SIZE):
for x in range(0, rgb_img.shape[1], TARGET_SIZE):
img_tile = rgb_img[y:y + TARGET_SIZE, x:x + TARGET_SIZE]
mask_tile = gt[y:y + TARGET_SIZE, x:x + TARGET_SIZE]
if img_tile.shape[0] == TARGET_SIZE and img_tile.shape[1] == TARGET_SIZE:
if not (mask_tile.mean() == 0 or mask_tile.mean() == 2) :
out_img_path = os.path.join(OUTPUT_DIR, "img_{}.png".format(k))
io.imsave(out_img_path, img_tile)
out_mask_path = os.path.join(OUTPUT_DIR, "msk_{}.png".format(k))
cv2.imwrite(out_mask_path, mask_tile)
k += 1
/usr/local/lib/python3.10/dist-packages/skimage/_shared/utils.py:328: UserWarning: /content/output/img_8.png is a low contrast image return func(*args, **kwargs) /usr/local/lib/python3.10/dist-packages/skimage/_shared/utils.py:328: UserWarning: /content/output/img_142.png is a low contrast image return func(*args, **kwargs) /usr/local/lib/python3.10/dist-packages/skimage/_shared/utils.py:328: UserWarning: /content/output/img_163.png is a low contrast image return func(*args, **kwargs) /usr/local/lib/python3.10/dist-packages/skimage/_shared/utils.py:328: UserWarning: /content/output/img_175.png is a low contrast image return func(*args, **kwargs)
Now let's create the X variable to receive the RGB and Y patches for the mask patches.
In [ ]:
list_img = [f for f in os.listdir(OUTPUT_DIR) if f.startswith('img')]
list_msk = [f for f in os.listdir(OUTPUT_DIR) if f.startswith('msk')]
In [ ]:
list_img.sort()
list_msk.sort()
In [ ]:
X = []
for path_img in list_img:
full_path = os.path.join(OUTPUT_DIR, path_img)
img = io.imread(full_path)
X.append(img)
In [ ]:
Y = []
for path_msk in list_msk:
full_path = os.path.join(OUTPUT_DIR, path_msk)
msk = io.imread(full_path)
Y.append(msk)
In [ ]:
X = np.array(X)
Y = np.array(Y)
In [ ]:
X.shape
Out[ ]:
(342, 256, 256, 4)
Let's visualize the data:
In [ ]:
i = 54
plt.figure(figsize=[10,10])
plt.subplot(121)
plt.imshow(X[i,:,:,:])
plt.title('RGB Image')
plt.axis('off')
plt.subplot(122)
plt.imshow(Y[i,:,:])
plt.title('True Image')
plt.axis('off')
Out[ ]:
(-0.5, 255.5, 255.5, -0.5)
Let's apply the one hot encoder to the labels:
In [ ]:
from keras.models import Model
from keras.regularizers import l2
from keras.layers import *
from keras.models import *
import keras.backend as K
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator
#from tensorflow.keras.optimizers import Adam
from keras.optimizers import Adam
from tensorflow.keras.losses import Tversky
In [ ]:
Y = to_categorical(Y,len(class_names_list))
In [ ]:
Y.shape
Out[ ]:
(342, 256, 256, 4)
Here we will divide the dataset into training and testing:
In [ ]:
x_train, x_test, y_train, y_test = train_test_split(X, Y, test_size=0.3, random_state=10)
In [ ]:
x_train = x_train/255
x_test = x_test/255
In [ ]:
y_train = y_train.astype('float')
y_test = y_test.astype('float')
We have implemented data augmentation:
In [ ]:
img_datagen = ImageDataGenerator(
rotation_range=90,
vertical_flip = True,
horizontal_flip=True)
mask_datagen = ImageDataGenerator(
rotation_range=90,
vertical_flip = True,
horizontal_flip=True)
In [ ]:
img_datagen.fit(x_train, augment=True,seed=1200)
mask_datagen.fit(y_train, augment=True,seed=1200)
In [ ]:
train_generator=img_datagen.flow(x_train,y_train,batch_size=8,seed=1200)
In [ ]:
steps_per_epoch = len(x_train)//8
validation_steps = len(x_test)//8
In [ ]:
len(x_test)/8
Out[ ]:
12.875
In [ ]:
steps_per_epoch = len(x_train)//8
validation_steps = len(x_test)//8
In [ ]:
n_classe = 4
And then we can build DeepLabV3+:
In [ ]:
import keras
from keras import layers
from keras import ops
def convolution_block(
block_input,
num_filters=256,
kernel_size=3,
dilation_rate=1,
use_bias=False,
):
x = layers.Conv2D(
num_filters,
kernel_size=kernel_size,
dilation_rate=dilation_rate,
padding="same",
use_bias=use_bias,
kernel_initializer=keras.initializers.HeNormal(),
)(block_input)
x = layers.BatchNormalization()(x)
return ops.nn.relu(x)
def DilatedSpatialPyramidPooling(dspp_input):
dims = dspp_input.shape
x = layers.AveragePooling2D(pool_size=(dims[-3], dims[-2]))(dspp_input)
x = convolution_block(x, kernel_size=1, use_bias=True)
out_pool = layers.UpSampling2D(
size=(dims[-3] // x.shape[1], dims[-2] // x.shape[2]),
interpolation="bilinear",
)(x)
out_1 = convolution_block(dspp_input, kernel_size=1, dilation_rate=1)
out_6 = convolution_block(dspp_input, kernel_size=3, dilation_rate=6)
out_12 = convolution_block(dspp_input, kernel_size=3, dilation_rate=12)
out_18 = convolution_block(dspp_input, kernel_size=3, dilation_rate=18)
x = layers.Concatenate(axis=-1)([out_pool, out_1, out_6, out_12, out_18])
output = convolution_block(x, kernel_size=1)
return output
In [ ]:
def DeeplabV3Plus(image_size, num_classes):
model_input = keras.Input(shape=image_size)
preprocessed = keras.applications.resnet50.preprocess_input(model_input)
resnet50 = keras.applications.ResNet50(
weights="imagenet", include_top=False, input_tensor=preprocessed
)
x = resnet50.get_layer("conv4_block6_2_relu").output
x = DilatedSpatialPyramidPooling(x)
input_a = layers.UpSampling2D(
size=(image_size[0] // 4 // x.shape[1], image_size[1] // 4 // x.shape[2]),
interpolation="bilinear",
)(x)
input_b = resnet50.get_layer("conv2_block3_2_relu").output
input_b = convolution_block(input_b, num_filters=48, kernel_size=1)
x = layers.Concatenate(axis=-1)([input_a, input_b])
x = convolution_block(x)
x = convolution_block(x)
x = layers.UpSampling2D(
size=(image_size[0] // x.shape[1], image_size[1] // x.shape[2]),
interpolation="bilinear",
)(x)
model_output = layers.Conv2D(num_classes, kernel_size=(1, 1), activation='sigmoid', padding="same")(x)
return keras.Model(inputs=model_input, outputs=model_output)
model = DeeplabV3Plus(image_size=x_train.shape[1:], num_classes=n_classe)
model.compile(optimizer=Adam(learning_rate = 1e-5), loss = Tversky, metrics=[ 'accuracy'])
model.summary()
Model: "functional_1"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ Connected to ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩ │ input_layer_1 │ (None, 256, 256, 4) │ 0 │ - │ │ (InputLayer) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ get_item_3 (GetItem) │ (None, 256, 256) │ 0 │ input_layer_1[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ get_item_4 (GetItem) │ (None, 256, 256) │ 0 │ input_layer_1[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ get_item_5 (GetItem) │ (None, 256, 256) │ 0 │ input_layer_1[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ stack_1 (Stack) │ (None, 256, 256, 3) │ 0 │ get_item_3[0][0], │ │ │ │ │ get_item_4[0][0], │ │ │ │ │ get_item_5[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ add_1 (Add) │ (None, 256, 256, 3) │ 0 │ stack_1[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv1_pad (ZeroPadding2D) │ (None, 262, 262, 3) │ 0 │ add_1[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv1_conv (Conv2D) │ (None, 128, 128, 64) │ 9,472 │ conv1_pad[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv1_bn │ (None, 128, 128, 64) │ 256 │ conv1_conv[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv1_relu (Activation) │ (None, 128, 128, 64) │ 0 │ conv1_bn[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ pool1_pad (ZeroPadding2D) │ (None, 130, 130, 64) │ 0 │ conv1_relu[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ pool1_pool (MaxPooling2D) │ (None, 64, 64, 64) │ 0 │ pool1_pad[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block1_1_conv │ (None, 64, 64, 64) │ 4,160 │ pool1_pool[0][0] │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block1_1_bn │ (None, 64, 64, 64) │ 256 │ conv2_block1_1_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block1_1_relu │ (None, 64, 64, 64) │ 0 │ conv2_block1_1_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block1_2_conv │ (None, 64, 64, 64) │ 36,928 │ conv2_block1_1_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block1_2_bn │ (None, 64, 64, 64) │ 256 │ conv2_block1_2_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block1_2_relu │ (None, 64, 64, 64) │ 0 │ conv2_block1_2_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block1_0_conv │ (None, 64, 64, 256) │ 16,640 │ pool1_pool[0][0] │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block1_3_conv │ (None, 64, 64, 256) │ 16,640 │ conv2_block1_2_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block1_0_bn │ (None, 64, 64, 256) │ 1,024 │ conv2_block1_0_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block1_3_bn │ (None, 64, 64, 256) │ 1,024 │ conv2_block1_3_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block1_add (Add) │ (None, 64, 64, 256) │ 0 │ conv2_block1_0_bn[0][… │ │ │ │ │ conv2_block1_3_bn[0][… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block1_out │ (None, 64, 64, 256) │ 0 │ conv2_block1_add[0][0] │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block2_1_conv │ (None, 64, 64, 64) │ 16,448 │ conv2_block1_out[0][0] │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block2_1_bn │ (None, 64, 64, 64) │ 256 │ conv2_block2_1_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block2_1_relu │ (None, 64, 64, 64) │ 0 │ conv2_block2_1_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block2_2_conv │ (None, 64, 64, 64) │ 36,928 │ conv2_block2_1_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block2_2_bn │ (None, 64, 64, 64) │ 256 │ conv2_block2_2_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block2_2_relu │ (None, 64, 64, 64) │ 0 │ conv2_block2_2_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block2_3_conv │ (None, 64, 64, 256) │ 16,640 │ conv2_block2_2_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block2_3_bn │ (None, 64, 64, 256) │ 1,024 │ conv2_block2_3_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block2_add (Add) │ (None, 64, 64, 256) │ 0 │ conv2_block1_out[0][0… │ │ │ │ │ conv2_block2_3_bn[0][… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block2_out │ (None, 64, 64, 256) │ 0 │ conv2_block2_add[0][0] │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block3_1_conv │ (None, 64, 64, 64) │ 16,448 │ conv2_block2_out[0][0] │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block3_1_bn │ (None, 64, 64, 64) │ 256 │ conv2_block3_1_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block3_1_relu │ (None, 64, 64, 64) │ 0 │ conv2_block3_1_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block3_2_conv │ (None, 64, 64, 64) │ 36,928 │ conv2_block3_1_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block3_2_bn │ (None, 64, 64, 64) │ 256 │ conv2_block3_2_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block3_2_relu │ (None, 64, 64, 64) │ 0 │ conv2_block3_2_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block3_3_conv │ (None, 64, 64, 256) │ 16,640 │ conv2_block3_2_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block3_3_bn │ (None, 64, 64, 256) │ 1,024 │ conv2_block3_3_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block3_add (Add) │ (None, 64, 64, 256) │ 0 │ conv2_block2_out[0][0… │ │ │ │ │ conv2_block3_3_bn[0][… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2_block3_out │ (None, 64, 64, 256) │ 0 │ conv2_block3_add[0][0] │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block1_1_conv │ (None, 32, 32, 128) │ 32,896 │ conv2_block3_out[0][0] │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block1_1_bn │ (None, 32, 32, 128) │ 512 │ conv3_block1_1_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block1_1_relu │ (None, 32, 32, 128) │ 0 │ conv3_block1_1_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block1_2_conv │ (None, 32, 32, 128) │ 147,584 │ conv3_block1_1_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block1_2_bn │ (None, 32, 32, 128) │ 512 │ conv3_block1_2_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block1_2_relu │ (None, 32, 32, 128) │ 0 │ conv3_block1_2_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block1_0_conv │ (None, 32, 32, 512) │ 131,584 │ conv2_block3_out[0][0] │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block1_3_conv │ (None, 32, 32, 512) │ 66,048 │ conv3_block1_2_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block1_0_bn │ (None, 32, 32, 512) │ 2,048 │ conv3_block1_0_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block1_3_bn │ (None, 32, 32, 512) │ 2,048 │ conv3_block1_3_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block1_add (Add) │ (None, 32, 32, 512) │ 0 │ conv3_block1_0_bn[0][… │ │ │ │ │ conv3_block1_3_bn[0][… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block1_out │ (None, 32, 32, 512) │ 0 │ conv3_block1_add[0][0] │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block2_1_conv │ (None, 32, 32, 128) │ 65,664 │ conv3_block1_out[0][0] │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block2_1_bn │ (None, 32, 32, 128) │ 512 │ conv3_block2_1_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block2_1_relu │ (None, 32, 32, 128) │ 0 │ conv3_block2_1_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block2_2_conv │ (None, 32, 32, 128) │ 147,584 │ conv3_block2_1_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block2_2_bn │ (None, 32, 32, 128) │ 512 │ conv3_block2_2_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block2_2_relu │ (None, 32, 32, 128) │ 0 │ conv3_block2_2_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block2_3_conv │ (None, 32, 32, 512) │ 66,048 │ conv3_block2_2_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block2_3_bn │ (None, 32, 32, 512) │ 2,048 │ conv3_block2_3_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block2_add (Add) │ (None, 32, 32, 512) │ 0 │ conv3_block1_out[0][0… │ │ │ │ │ conv3_block2_3_bn[0][… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block2_out │ (None, 32, 32, 512) │ 0 │ conv3_block2_add[0][0] │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block3_1_conv │ (None, 32, 32, 128) │ 65,664 │ conv3_block2_out[0][0] │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block3_1_bn │ (None, 32, 32, 128) │ 512 │ conv3_block3_1_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block3_1_relu │ (None, 32, 32, 128) │ 0 │ conv3_block3_1_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block3_2_conv │ (None, 32, 32, 128) │ 147,584 │ conv3_block3_1_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block3_2_bn │ (None, 32, 32, 128) │ 512 │ conv3_block3_2_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block3_2_relu │ (None, 32, 32, 128) │ 0 │ conv3_block3_2_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block3_3_conv │ (None, 32, 32, 512) │ 66,048 │ conv3_block3_2_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block3_3_bn │ (None, 32, 32, 512) │ 2,048 │ conv3_block3_3_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block3_add (Add) │ (None, 32, 32, 512) │ 0 │ conv3_block2_out[0][0… │ │ │ │ │ conv3_block3_3_bn[0][… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block3_out │ (None, 32, 32, 512) │ 0 │ conv3_block3_add[0][0] │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block4_1_conv │ (None, 32, 32, 128) │ 65,664 │ conv3_block3_out[0][0] │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block4_1_bn │ (None, 32, 32, 128) │ 512 │ conv3_block4_1_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block4_1_relu │ (None, 32, 32, 128) │ 0 │ conv3_block4_1_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block4_2_conv │ (None, 32, 32, 128) │ 147,584 │ conv3_block4_1_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block4_2_bn │ (None, 32, 32, 128) │ 512 │ conv3_block4_2_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block4_2_relu │ (None, 32, 32, 128) │ 0 │ conv3_block4_2_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block4_3_conv │ (None, 32, 32, 512) │ 66,048 │ conv3_block4_2_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block4_3_bn │ (None, 32, 32, 512) │ 2,048 │ conv3_block4_3_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block4_add (Add) │ (None, 32, 32, 512) │ 0 │ conv3_block3_out[0][0… │ │ │ │ │ conv3_block4_3_bn[0][… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv3_block4_out │ (None, 32, 32, 512) │ 0 │ conv3_block4_add[0][0] │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block1_1_conv │ (None, 16, 16, 256) │ 131,328 │ conv3_block4_out[0][0] │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block1_1_bn │ (None, 16, 16, 256) │ 1,024 │ conv4_block1_1_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block1_1_relu │ (None, 16, 16, 256) │ 0 │ conv4_block1_1_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block1_2_conv │ (None, 16, 16, 256) │ 590,080 │ conv4_block1_1_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block1_2_bn │ (None, 16, 16, 256) │ 1,024 │ conv4_block1_2_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block1_2_relu │ (None, 16, 16, 256) │ 0 │ conv4_block1_2_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block1_0_conv │ (None, 16, 16, 1024) │ 525,312 │ conv3_block4_out[0][0] │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block1_3_conv │ (None, 16, 16, 1024) │ 263,168 │ conv4_block1_2_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block1_0_bn │ (None, 16, 16, 1024) │ 4,096 │ conv4_block1_0_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block1_3_bn │ (None, 16, 16, 1024) │ 4,096 │ conv4_block1_3_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block1_add (Add) │ (None, 16, 16, 1024) │ 0 │ conv4_block1_0_bn[0][… │ │ │ │ │ conv4_block1_3_bn[0][… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block1_out │ (None, 16, 16, 1024) │ 0 │ conv4_block1_add[0][0] │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block2_1_conv │ (None, 16, 16, 256) │ 262,400 │ conv4_block1_out[0][0] │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block2_1_bn │ (None, 16, 16, 256) │ 1,024 │ conv4_block2_1_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block2_1_relu │ (None, 16, 16, 256) │ 0 │ conv4_block2_1_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block2_2_conv │ (None, 16, 16, 256) │ 590,080 │ conv4_block2_1_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block2_2_bn │ (None, 16, 16, 256) │ 1,024 │ conv4_block2_2_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block2_2_relu │ (None, 16, 16, 256) │ 0 │ conv4_block2_2_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block2_3_conv │ (None, 16, 16, 1024) │ 263,168 │ conv4_block2_2_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block2_3_bn │ (None, 16, 16, 1024) │ 4,096 │ conv4_block2_3_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block2_add (Add) │ (None, 16, 16, 1024) │ 0 │ conv4_block1_out[0][0… │ │ │ │ │ conv4_block2_3_bn[0][… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block2_out │ (None, 16, 16, 1024) │ 0 │ conv4_block2_add[0][0] │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block3_1_conv │ (None, 16, 16, 256) │ 262,400 │ conv4_block2_out[0][0] │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block3_1_bn │ (None, 16, 16, 256) │ 1,024 │ conv4_block3_1_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block3_1_relu │ (None, 16, 16, 256) │ 0 │ conv4_block3_1_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block3_2_conv │ (None, 16, 16, 256) │ 590,080 │ conv4_block3_1_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block3_2_bn │ (None, 16, 16, 256) │ 1,024 │ conv4_block3_2_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block3_2_relu │ (None, 16, 16, 256) │ 0 │ conv4_block3_2_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block3_3_conv │ (None, 16, 16, 1024) │ 263,168 │ conv4_block3_2_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block3_3_bn │ (None, 16, 16, 1024) │ 4,096 │ conv4_block3_3_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block3_add (Add) │ (None, 16, 16, 1024) │ 0 │ conv4_block2_out[0][0… │ │ │ │ │ conv4_block3_3_bn[0][… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block3_out │ (None, 16, 16, 1024) │ 0 │ conv4_block3_add[0][0] │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block4_1_conv │ (None, 16, 16, 256) │ 262,400 │ conv4_block3_out[0][0] │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block4_1_bn │ (None, 16, 16, 256) │ 1,024 │ conv4_block4_1_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block4_1_relu │ (None, 16, 16, 256) │ 0 │ conv4_block4_1_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block4_2_conv │ (None, 16, 16, 256) │ 590,080 │ conv4_block4_1_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block4_2_bn │ (None, 16, 16, 256) │ 1,024 │ conv4_block4_2_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block4_2_relu │ (None, 16, 16, 256) │ 0 │ conv4_block4_2_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block4_3_conv │ (None, 16, 16, 1024) │ 263,168 │ conv4_block4_2_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block4_3_bn │ (None, 16, 16, 1024) │ 4,096 │ conv4_block4_3_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block4_add (Add) │ (None, 16, 16, 1024) │ 0 │ conv4_block3_out[0][0… │ │ │ │ │ conv4_block4_3_bn[0][… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block4_out │ (None, 16, 16, 1024) │ 0 │ conv4_block4_add[0][0] │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block5_1_conv │ (None, 16, 16, 256) │ 262,400 │ conv4_block4_out[0][0] │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block5_1_bn │ (None, 16, 16, 256) │ 1,024 │ conv4_block5_1_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block5_1_relu │ (None, 16, 16, 256) │ 0 │ conv4_block5_1_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block5_2_conv │ (None, 16, 16, 256) │ 590,080 │ conv4_block5_1_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block5_2_bn │ (None, 16, 16, 256) │ 1,024 │ conv4_block5_2_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block5_2_relu │ (None, 16, 16, 256) │ 0 │ conv4_block5_2_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block5_3_conv │ (None, 16, 16, 1024) │ 263,168 │ conv4_block5_2_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block5_3_bn │ (None, 16, 16, 1024) │ 4,096 │ conv4_block5_3_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block5_add (Add) │ (None, 16, 16, 1024) │ 0 │ conv4_block4_out[0][0… │ │ │ │ │ conv4_block5_3_bn[0][… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block5_out │ (None, 16, 16, 1024) │ 0 │ conv4_block5_add[0][0] │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block6_1_conv │ (None, 16, 16, 256) │ 262,400 │ conv4_block5_out[0][0] │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block6_1_bn │ (None, 16, 16, 256) │ 1,024 │ conv4_block6_1_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block6_1_relu │ (None, 16, 16, 256) │ 0 │ conv4_block6_1_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block6_2_conv │ (None, 16, 16, 256) │ 590,080 │ conv4_block6_1_relu[0… │ │ (Conv2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block6_2_bn │ (None, 16, 16, 256) │ 1,024 │ conv4_block6_2_conv[0… │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv4_block6_2_relu │ (None, 16, 16, 256) │ 0 │ conv4_block6_2_bn[0][… │ │ (Activation) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ average_pooling2d_1 │ (None, 1, 1, 256) │ 0 │ conv4_block6_2_relu[0… │ │ (AveragePooling2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_10 (Conv2D) │ (None, 1, 1, 256) │ 65,792 │ average_pooling2d_1[0… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_9 │ (None, 1, 1, 256) │ 1,024 │ conv2d_10[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_11 (Conv2D) │ (None, 16, 16, 256) │ 65,536 │ conv4_block6_2_relu[0… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_12 (Conv2D) │ (None, 16, 16, 256) │ 589,824 │ conv4_block6_2_relu[0… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_13 (Conv2D) │ (None, 16, 16, 256) │ 589,824 │ conv4_block6_2_relu[0… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_14 (Conv2D) │ (None, 16, 16, 256) │ 589,824 │ conv4_block6_2_relu[0… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ relu_9 (Relu) │ (None, 1, 1, 256) │ 0 │ batch_normalization_9… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_10 │ (None, 16, 16, 256) │ 1,024 │ conv2d_11[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_11 │ (None, 16, 16, 256) │ 1,024 │ conv2d_12[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_12 │ (None, 16, 16, 256) │ 1,024 │ conv2d_13[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_13 │ (None, 16, 16, 256) │ 1,024 │ conv2d_14[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ up_sampling2d_3 │ (None, 16, 16, 256) │ 0 │ relu_9[0][0] │ │ (UpSampling2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ relu_10 (Relu) │ (None, 16, 16, 256) │ 0 │ batch_normalization_1… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ relu_11 (Relu) │ (None, 16, 16, 256) │ 0 │ batch_normalization_1… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ relu_12 (Relu) │ (None, 16, 16, 256) │ 0 │ batch_normalization_1… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ relu_13 (Relu) │ (None, 16, 16, 256) │ 0 │ batch_normalization_1… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ concatenate_2 │ (None, 16, 16, 1280) │ 0 │ up_sampling2d_3[0][0], │ │ (Concatenate) │ │ │ relu_10[0][0], │ │ │ │ │ relu_11[0][0], │ │ │ │ │ relu_12[0][0], │ │ │ │ │ relu_13[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_15 (Conv2D) │ (None, 16, 16, 256) │ 327,680 │ concatenate_2[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_14 │ (None, 16, 16, 256) │ 1,024 │ conv2d_15[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_16 (Conv2D) │ (None, 64, 64, 48) │ 3,072 │ conv2_block3_2_relu[0… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ relu_14 (Relu) │ (None, 16, 16, 256) │ 0 │ batch_normalization_1… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_15 │ (None, 64, 64, 48) │ 192 │ conv2d_16[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ up_sampling2d_4 │ (None, 64, 64, 256) │ 0 │ relu_14[0][0] │ │ (UpSampling2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ relu_15 (Relu) │ (None, 64, 64, 48) │ 0 │ batch_normalization_1… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ concatenate_3 │ (None, 64, 64, 304) │ 0 │ up_sampling2d_4[0][0], │ │ (Concatenate) │ │ │ relu_15[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_17 (Conv2D) │ (None, 64, 64, 256) │ 700,416 │ concatenate_3[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_16 │ (None, 64, 64, 256) │ 1,024 │ conv2d_17[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ relu_16 (Relu) │ (None, 64, 64, 256) │ 0 │ batch_normalization_1… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_18 (Conv2D) │ (None, 64, 64, 256) │ 589,824 │ relu_16[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_17 │ (None, 64, 64, 256) │ 1,024 │ conv2d_18[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ relu_17 (Relu) │ (None, 64, 64, 256) │ 0 │ batch_normalization_1… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ up_sampling2d_5 │ (None, 256, 256, 256) │ 0 │ relu_17[0][0] │ │ (UpSampling2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_19 (Conv2D) │ (None, 256, 256, 4) │ 1,028 │ up_sampling2d_5[0][0] │ └───────────────────────────┴────────────────────────┴────────────────┴────────────────────────┘
Total params: 11,853,124 (45.22 MB)
Trainable params: 11,820,388 (45.09 MB)
Non-trainable params: 32,736 (127.88 KB)
In [ ]:
history = model.fit(train_generator,steps_per_epoch=steps_per_epoch, validation_steps=validation_steps,
epochs=300, validation_data=(x_test,y_test))
Epoch 1/300
/usr/local/lib/python3.10/dist-packages/keras/src/trainers/data_adapters/py_dataset_adapter.py:122: UserWarning: Your `PyDataset` class should call `super().__init__(**kwargs)` in its constructor. `**kwargs` can include `workers`, `use_multiprocessing`, `max_queue_size`. Do not pass these arguments to `fit()`, as they will be ignored. self._warn_if_super_not_called()
29/29 ━━━━━━━━━━━━━━━━━━━━ 638s 20s/step - accuracy: 0.2491 - loss: 0.6470 - val_accuracy: 0.0711 - val_loss: 0.6793 Epoch 2/300 1/29 ━━━━━━━━━━━━━━━━━━━━ 8:18 18s/step - accuracy: 0.6227 - loss: 0.5518
/usr/lib/python3.10/contextlib.py:153: UserWarning: Your input ran out of data; interrupting training. Make sure that your dataset or generator can generate at least `steps_per_epoch * epochs` batches. You may need to use the `.repeat()` function when building your dataset. self.gen.throw(typ, value, traceback)
29/29 ━━━━━━━━━━━━━━━━━━━━ 34s 572ms/step - accuracy: 0.6227 - loss: 0.5518 Epoch 3/300 2/29 ━━━━━━━━━━━━━━━━━━━━ 8:22 19s/step - accuracy: 0.4699 - loss: 0.6014
In [ ]:
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'val'], loc='lower right')
plt.show()
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper right')
plt.show()
After training, we will obtain some validation metrics:
In [ ]:
from keras.models import model_from_json
from sklearn.metrics import classification_report
from sklearn.metrics import confusion_matrix
import pandas as pd
import seaborn as sns
from sklearn.metrics import f1_score
from sklearn.metrics import recall_score
from sklearn.metrics import cohen_kappa_score
We start by calculating the accuracy for the test set:
In [ ]:
predict = model.predict(x_test)
4/4 [==============================] - 0s 108ms/step
In [ ]:
predict = np.round(predict)
In [ ]:
pred = np.argmax(predict, axis=3)
In [ ]:
true = np.argmax(y_test, axis=3)
Accuracy:
In [ ]:
accuracy = accuracy_score(true.flatten(),pred.flatten())
print(accuracy)
0.9479219010732707
F1 Score:
In [ ]:
f1_score(true.flatten(), pred.flatten(), average='macro')
Out[ ]:
0.7527458780899748
In [ ]:
f1_score(true.flatten(), pred.flatten(), average='micro')
Out[ ]:
0.9479219010732707
In [ ]:
f1_score(true.flatten(), pred.flatten(), average='weighted')
Out[ ]:
0.9395697605269571
Recall:
In [ ]:
recall_score(true.flatten(), pred.flatten(), average='macro')
Out[ ]:
0.7117527073740617
Kappa
In [ ]:
cohen_kappa_score(true.flatten(), pred.flatten())
Out[ ]:
0.8385617288552756
Confusion Matrix:
In [ ]:
cm = confusion_matrix(true.flatten(), pred.flatten())
print(cm)
[[ 875568 0 381 175] [ 9519 86870 161540 32956] [ 28130 21241 5326131 11174] [ 5044 107 81271 110101]]
In [ ]:
columns = class_names_list
r1 = pd.DataFrame(data=cm, columns=columns, index=columns)
fig, ax = plt.subplots(figsize=(10,8))
ax = sns.heatmap(r1, annot=True, annot_kws={"size": 18},fmt='d',cmap="Blues", cbar = False)
#for t in ax.texts: t.set_text(t.get_text() + " %")
ax.tick_params(labelsize=16)
ax.set_ylabel('Verdadeiro')
ax.set_xlabel('Predito')
Out[ ]:
Text(0.5, 58.5815972222222, 'Predito')
Finally, let's plot an example of the predicted result compared to the original mask:
In [ ]:
i = 10
plt.figure(figsize=[20,8])
plt.subplot(131)
plt.imshow(x_test[i])
plt.title('RGB Image')
plt.axis('off')
plt.subplot(132)
plt.imshow(true[i], vmin=0,vmax=3)
plt.title('Original Mask')
plt.axis('off')
plt.subplot(133)
plt.imshow(pred[i], vmin=0,vmax=3)
plt.title('Predict Mask')
plt.axis('off')
Out[ ]:
(-0.5, 255.5, 255.5, -0.5)
We can also save the trained architecture and weights:
In [ ]:
# serialize model to JSON
model_json = model.to_json()
with open("/content/drive/MyDrive/Datasets/Cana weeds/Deeplabv3.json", "w") as json_file:
json_file.write(model_json)
# serialize weights to HDF5
model.save_weights("/content/drive/MyDrive/Datasets/Cana weeds/Model300.weights.h5")